LAYERED MODULATION FOR TERRESTRIAL ATSC APPLICATIONS 

CROSS-REFERENCE TO RELATED APPLICATIONS 



[0001] This application claims the benefit of U.S. Provisional Patent Application No. 
60/421,327, entitled "LAYERED MODULATION FOR ATSC APPLICATIONS," by Ernest 
C. Chen, filed October 25, 2002. 

[0002] This application is also a continuation-in-part application of the following co- 
pending and commonly- assigned U.S. utility patent application, which applications is 
incorporated by reference herein: 

[0003] U.S. Utility Application Serial No. 09/844,401, filed April 27, 2001, by Emest C. 
Chen, entitled "LAYERED MODULATION FOR DIGITAL SIGNALS," attorneys' docket 
number PD-2001 81 (1 09.005 1-US-Ol). 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0004] The present invention relates to systems and methods for transmitting digital signals 
in ATSC applications, particularly signals using layered modulations. 

2. Description of the Related Art 

[0005] Digital signal communication systems have been used in various fields, including 
digital TV signal transmission, either terrestrial or satellite. As the various digital signal 
communication systems and services evolve, there is a burgeoning demand for increased data 
throughput and added services. However, it is more difficult to implement either 
improvement in old systems and new services when it is necessary to replace existing legacy 
hardware, such as transmitters and receivers. New systems and services are advantaged when 
they can utilize existing legacy hardware. In the realm of wireless communications, this 
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principle is further highlighted by the limited availability of electromagnetic spectrum. Thus, 
it is not possible (or at least not practical) to merely transmit enhanced or additional data at a 
new frequency. 

[0006] The conventional method of increasing spectral capacity is to move to a higher- 
order modulation, such as from quadrature phase shift keying (QPSK) to eight phase shift 
keying (8PSK) or sixteen quadrature amplitude modulation (16QAM). Unfortunately, QPSK 
receivers cannot demodulate conventional 8PSK or 16QAM signals. As a result, legacy 
customers with QPSK receivers must upgrade their receivers in order to continue to receive 
any signals transmitted with an 8PSK or 1 6QAM modulation. 

[0007] It is advantageous for systems and methods of transmitting signals to accommodate 
enhanced and increased data throughput without requiring additional frequency. In addition, 
it is advantageous for enhanced and increased throughput signals for new receivers to be 
backwards compatible with legacy receivers. There is further an advantage for systems and 
methods which allow transmission signals to be upgraded from a source separate from the 
legacy transmitter. 

[0008] It has been proposed that a layered modulation signal, transmitting non-coherently 
(asynchronously) both upper and lower layer signals, can be employed to meet these needs. 
Such layered modulation systems allow higher information throughput with backwards 
compatibility. Even when backward compatibility is not required (such as with an entirely 
new system), layered modulation can still be advantageous because it requires a TWTA peak 
power significantly lower than that for a conventional 8PSK or 16QAM modulation format 
for a given throughput. 

[0009] The Advanced Television Systems Committee, Inc., is an intemational, non-profit 
organization developing voluntary standards for digital television. The ATSC member 
organizations represent the broadcast, broadcast equipment, motion picture, consumer 
electronics, computer, cable, satellite, and semiconductor industries. The ATSC works to 
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coordinate television standards among different communications media focusing on digital 
television, interactive systems, and broadband multimedia communications. ATSC also 
develops digital television implementation strategies and present educational seminars on the 
ATSC standards. ATSC Digital TV Standards include digital high definition television 
(HDTV), standard definition television (SDTV), data broadcasting, multichannel surround- 
sound audio, and satellite direct-to-home broadcasting. 

[0010] For example, the terrestrial ATSC describes the characteristics of an 
RP/transmission subsystem, which is referred to as the VSB subsystem, of the Digital 
Television Standard. The VSB subsystem offers two modes: a terrestrial broadcast mode (8 
VSB), and a high data rate mode (16 VSB). See "ATSC Standard: Digital Television 
Standard, Revision B with Amendments 1 and 2", The Advanced Television Systems 
Committee, May 19, 2003, which is incorporated by reference herein. In the ATSC standard 
for digital television and many other communications standards, the allocated frequency 
bandwidth cannot be increased. Given the stipulated ATSC modulation and coding 
techniques, the information throughput for ATSC is fixed over the chaimel bandwidth of 6 
MHz. 

[0011] Accordingly, there is a need for systems and methods that expand capacity of the 
allocated ATSC frequency bandwidth. As discussed hereafter, the present invention meets 
these needs. 

SUMMARY OF THE INVENTION 

[0012] Layered modulation transmits two or more signals simultaneously. The signals are 
layered in terms of power level and are generated non-coherently with respect to each other. 
Upon reception, these signals are processed into separate data transports for a combined 
throughput greater than that afforded by the conventional modulation method which allows 
only one layer of signal. Embodiments of the present invention utilize layered modulation to 
provide additional throughput over the ATSC signal band. 
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[0013] Like other layered modulation applications, embodiments of the invention allow a 
new service to be added to existing service while maintaining backward compatibility. 
Specifically, performance of the legacy ATSC service with existing receivers will be little 
affected, and little extra power will be required to transmit the legacy signal. The signal for 
the new service will be transmitted typically from a separate antenna. The maximum range 
of the new service will be shorter than that of the legacy service, as will become clear later in 
this document. 

[0014] A typical transmission system of the invention includes a first antenna for 
transmitting an upper layer signal comprising an 8-VSB signal and a second antenna for 
transmitting a lower layer signal. A layered modulation signal comprises the upper layer 
signal and the lower layer signal both interfering in a common frequency band. At least one 
receiver demodulates the upper layer signal directly from the layered modulation signal and 
demodulates the lower layer signal after subtracting the upper layer signal from the layered 
modulation signal. The second antenna can have a selectively limited range so that the lower 
layer signal does not interfere with the upper layer signal in a range where the lower layer 
signal could not be demodulated. 

[0015] The selectively limited range can be produced by reducing a second antenna height 

ft 

relative to a first antenna height. Altemately or in combination with the reduced second 
antenna height, the second antenna can be a shaped-beam antenna in order to produce the 
selectively limited range. 

[0016] In a typical embodiment, the upper layer signal comprises a legacy signal and the 
lower layer signal comprises a 2-VSB signal. However, the lower layer signal can altemately 
be a QPSK or other signal. 

[0017] A typical receiver of the invention includes a first demodulator for demodulating an 
upper layer signal comprising an 8-VSB signal directly from a layered modulation signal and 
a second demodulator for demodulating a lower layer signal after subtracting the upper layer 
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signal from the layered modulation signal. The layered modulation signal comprises both the 
upper layer signal and a lower layer signal both interfering in a common frequency band. 

[0018] The upper layer signal can be subtracted from the layered modulation signal with a 
carrier of the upper layer signal included in the subtraction. Alternately, the upper layer 
signal can be subtracted from the layered modulation signal with the carrier of the upper layer 
signal removed before the subtraction. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0019] Referring now to the drawings in which like reference numbers represent 
corresponding parts throughout: 

[0020] FIG. 1 shows a functional block diagram of a representative terrestrial broadcast 
transmitter for the ATSC digital TV standard; 

[0021] FIG. 2 shows ATSC digital TV data organization for transmission; 
[0022] FIG. 3 illustrates a VSB channel spectrum; 

[0023] FIGS. 4A-4D illustrates the relationship between the upper layer legacy 8- VSB 
symbol constellation and the lower layer 2- VSB signal; 

[0024] FIG. 5 illustrates the geometry used to calculate antenna gain for exemplary CNR 
performance curves; 

[0025] FIGS. 6 A & 6B illustrate an exemplary CNR with the insertion of a lower layer 
VSB signal over the legacy VSB signal; 

[0026] FIG. 7 illustrates a exemplary system architecture of the invention; 

[0027] FIGS. 8A & SB illustrate exemplary receivers of the invention; and 

[0028] FIGS. 9A & 9B illustrates exemplary methods of the invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 



[0029] In the following description of the preferred embodiment, reference is made to the 
accompanying drawings which form a part hereof, and in which is shown by way of 
illustration a specific embodiment in which the invention may be practiced. It is to be 
understood that other embodiments may be utilized and structural changes may be made 
without departing from the scope of the present invention. 

1 . ATSC Terrestrial Transmission Characteristics 

10030] FIG. 1 shows a functional block diagram of a representative terrestrial broadcast 
transmitter 100 for the ATSC advanced television (ATV) digital standard. The ATSC uses a 
terrestrial broadcast mode (known as 8 VSB) to support a payload data rate of 19.28 Mbps in 
a 6 MHz channel. The input to the transmission subsystem from the transport subsystem is a 
19.39 Mbps serial data stream comprising 188-byte MPEG-compatible data packets 
(including a sync byte and 187 bytes of data which represent a payload data rate of 19.28 
Mbps). 

[0031] The incoming data is randomized and then processed for forward error correction 
(FEC) in the form of Reed-Solomon (RS) coding (20 RS parity bytes are added to each 
packet), 1/6 data field interleaving and 2/3 rate trellis coding. The randomization and FEC 
processes are not applied to the sync byte of the transport packet, which is represented in 
transmission by a data segment sync signal as described below. Following randomization 
and forward error correction processing, the data packets are formatted into Data Frames for 
transmission and Data Segment Sync and Data Field Sync are added. 

[0032] FIG. 2 shows how the data are organized for transmission. Each data frame 
comprises two data fields, each containing 313 data segments. The first data segment of each 
data field is a unique synchronizing signal (data field sync) and includes the training 
sequence used by the equalizer in the receiver. The remaining 312 data segments each carry 
the equivalent of the data from one 188-byte transport packet plus its associated FEC 
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overhead. The actual data in each data segment comes from several transport packets 
because of data interleaving. Each data segment consists of 832 symbols. The first 4 
symbols are transmitted in binary form and provide segment synchronization. This data 
segment sync signal also represents the sync b>te of the 18 8-byte MPEG-compatible 
transport packet. The remaining 828 symbols of each data segment carry data equivalent to 
the remaining 187 bytes of a transport packet and its associated EEC overhead. These 828 
symbols are transmitted as 8-level signals and therefore carry three bits per symbol. Thus, 
828 X 3 = 2484 bits of data are carried in each data segment, which exactly matches the 
requirement to send a protected transport packet: 187 data bytes + 20 RS parity bytes = 207 
bytes, 207 bytes x 8 bits/byte = 1656 bits and 2/3 rate trellis coding requires 3/2 x 1656 bits = 

4 5 

2484 bits. The symbol rate in MHz is given by S^{MHz) = x 684 = 10.76 , which is twice 

286 

of 5.38 MHz for a single-sideband (SSB) signal The frequency of a data segment (in data 

segments/s) is given by /^^^ = — ^ = 12.94 x 10\ The data frame rate (in frames/s) is given 

832 

fse 

by ff^a^^ = = 20.66 . The symbol rate and the transport rate are locked to each other in 

626 

frequency. 

[0033] FIG. 3 shows the nominal VSB transmission spectrum. The 8-level symbols 
combined with the binary data segment sync and data field sync signals shall be used to 
suppressed-carrier modulate a single carrier. Before transmission, however, most of the 
lower sideband shall be removed. The resulting spectrum is flat, except for the band edges 
where a nominal square root raised cosine response results in 620 kHz transition regions. At 
the suppressed-carrier frequency, 310 kHz from the lower band edge, a small pilot 300 is 
added to the signal. 

[0034] Referring back to FIG. 1, all payload data is carried with the same priority. A data 
randomizer 102 is used on all input data to randomize the data payload (not including data 
field sync or data segment sync, or RS parity bytes). The data randomizer 102 XORs all the 
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incoming data bytes with a 16-bit maximum length pseudo random binary sequence (PRBS) 
which is initiaUzed at the beginning of the data field. The PRBS is generated in a 16-bit shift 
register that has nine feedback taps. Eight of the shift register outputs are selected as the 
fixed randomizing byte, where each bit fi-om this byte is used to individually XOR the 
corresponding input data bit. The data bits are XORed most significant bit (MSB) to MSB, 
least significant bit (LSB ) to LSB, etc. The data randomizer 1 02 generator polynomial is 

^(16) X'^ + X'' X'^ ^ X'' -\- X' + X' X' X -^l . The initialization (pre-load) of the 

randomizer to F 180 hex (load to 1) occurs during the data segment sync interval prior to the 
first data segment. 

[0035] The data randomizer 102 is coupled to a Reed-Soloman encoder 104. The RS code 
used in the VSB transmission subsystem shall be a t = 10 (207,187) code. The RS data block 
size is 187 bytes, with 20 RS parity bytes added for error correction. A total RS block size of 
207 bytes is transmitted per data segment. Li creating bytes fi-om the serial bit stream, the 
MSB shall be the first serial bit. The 20 RS parity bytes shall be sent at the end of the data 
segment. 

[0036] The interleaver 106 employed in the VSB transmitter 100 can be a 52 data segment 
(intersegment) convolutional byte interleaver 106. Interleaving is provided to a depth of 
about 1/6 of a data field (4 ms deep). Only data bytes are interleaved. The interleaver shall 
be synchronized to the first data byte of the data field. Intrasegment interleaving is also 
performed for the benefit of the trellis coding process performed later. 

[0037] Following the interleaver 106, data is passed to the trellis encoder 108 which 
includes a precoder and symbol mapper. The 8 VSB trellis encoder 1 08 employs a 2/3 rate 
(R = 2/3) trellis code (with one unencoded bit which is precoded). That is, one input bit is 
encoded into two output bits using a 1/2 rate convolutional code while the other input bit is 
precoded. The signaling waveform used with the trellis code is an 8-level (3 bit) one- 
dimensional constellation. Thus, the transmitted signal is referred to as 8 VSB. A four-state 
trellis encoder is used and trellis code intrasegment interleaving is used. This employs twelve 
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identical trellis encoders 108 and precoders operating on interleaved data symbols. (The 
overall trellis encoder and the individual 12 encoder are both referenced as 108.) The code 
interleaving is accomplished by encoding symbols (0, 12, 24, 36 ...) as one group, symbols 
(1, 13, 25, 37, ...) as a second group, symbols (2, 14, 26, 38, ...) as a third group, and so on for 
a total of 12 groups. 

[0038] In creating serial bits from parallel bytes, the MSB is sent out first: (7, 6, 5, 4, 3, 2, 

1, 0). The MSB is precoded (7, 5, 3, 1) and the LSB is feedback convolutional encoded (6, 4, 

2, 0). Standard four-state optimal Ungerboeck codes shall be used for the encoding. The 
trellis code utilizes the four-state feedback encoder. The trellis code and precoder 
intrasegment interleaver feeds the symbol mapper. Data bytes are fed from the byte 
interleaver 106 to the trellis encoder 108 and precoder, and they are processed as whole bytes 
by each of the twelve encoders 108. Each byte produces four symbols from a single encoder 
108. The value of 1.25 is added to all these nominal levels after the bit-to-symbol mapping 
function for the purpose of creating a small pilot carrier, discussed hereafter. 

[0039] Following the trellis encoder 108 the output multiplexer 110 advances by four 
symbols on each segment boundary. However, the state of the trellis encoder 108 shall not be 
advanced. The data coming out of the multiplexer 110 shall follow normal ordering from 
encoder 0 through 1 1 of the trellis encoder 108 for the first segment of the frame, but on the 
second segment the order changes and symbols are read from encoders 4 through 1 1 , and 
then 0 through 3 of the trellis encoder 108. The third segment reads from encoder 8 through 
1 1 and then encoder 0 through 7 of the trellis encoder 108. This three-segment pattem shall 
repeat through the 312 data segments of the frame. After the data segment sync 1 12 is 
inserted, the ordering of the data symbols is such that symbols from each encoder 108 occur 
at a spacing of twelve symbols. 

[0040] A complete conversion of parallel bytes to serial bits needs 828 bytes to produce 
6624 bits. Data symbols are created from 2 bits sent in MSB order, so a complete conversion 
operation yields 3312 data symbols, which corresponds to 4 segments of 828 data symbols. 
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3312 data symbols divided by 12 trellis encoders 108 gives 276 symbols per trellis encoder 
108. 276 symbols divided by 4 symbols per byte gives 69 bytes per trellis encoder 108. The 
conversion starts with the first segment of the field and proceeds with groups of 4 segments 
until the end of the field. The 312 segments per field divided by 4 gives 78 conversion 
operations per field. During segment sync 112 the input to 4 encoders 108 is skipped and the 
trellis encoders 108 cycle with no input. The input is held until the next multiplex cycle and 
then fed to the correct trellis encoder 108. 

[0041] The encoded trellis data shall be passed through a multiplexer 110 that inserts the 
various synchronization signals (data segment sync 112 and data field sync 114). A two- 
level (binary) four-symbol data segment sync 1 12 is inserted into the 8-level digital data 

stream at the beginning of each data segment. (The MPEG sync byte shall be replaced by 
data segment sync 112.) 

[0042] A complete data segment comprises 832 symbols: 4 symbols for data segment sync 
112, and 828 data plus parity symbols. The data segment sync 1 12 is binary (2-level). The 
same sync pattem occurs regularly at 77.3 jxs intervals, and is the only signal repeating at this 
rate. Unlike the data, the four symbols for the data segment sync 112 are not Reed-Solomon, 
trellis encoded or interleaved. The data segment sync 112 pattem is a 1001 pattem. 

[0043] The data are not only divided into data segments, but also into data fields, each 
comprising 313 segments. Each data field (24.2 ms) begins with one complete data segment 
of data field sync 114. Each symbol represents one bit of data (2-level). Like the data 
segment sync 112, the field sync 1 14 is not Reed-Solomon, trellis encoded or interleaved. 

[0044] Following the multiplexer 1 10, a small in-phase pilot signal 300 is added to the data 
signal in the pilot insertion block 116. The fi-equency of the pilot signal 300 shall be the 
same as the suppressed-carrier frequency as shown in FIG. 3. This may be generated in the 
following manner. A small (digital) dc level (e.g. 1.25 V) can be added to every symbol 
(data and sync) of the digital baseband data plus sync signal (+1, +3, +5, +7). The power of 
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the pilot signal 300 is approximately 1 1.3 dB below the average data signal power. 
Following the pilot insertion block 116, the signal can be optionally passed to a pre-equalizer 
filter 118. 

[0045] Following the pilot insertion block 116 (and optional pre-equalizer filter 1 1 8), the 
VSB modulator 120 receives the 10.76 Msymbols/s, 8-level trellis encoded composite data 
signal (with the pilot and sync signals added). The ATV system performance is based on a 
linear phase raised cosine Nyquist filter response in the concatenated transmitter 100 and 
receiver, as shown in FIG. 3. The system filter response is essentially flat across the entire 
band, except for the transition regions at each end of the band. Nominally, the roll-off in the 
transmitter 1 00 shall have the response of a linear phase root raised cosine filter. As known 
in the art, the conventional receiving process of the 8 -VSB essentially performs the reverse of 
the transmission processes in a reverse order. 

2. Layered Modulation in Advanced Television Systems Committee (ATSC) 
Applications 

[0046] Embodiments of the present invention apply the principle of layered modulation, as 
detailed in U.S. Utility Application Serial No. 09/844,401, filed April 27, 2001, by Ernest C. 
Chen, entitled "LAYERED MODULATION FOR DIGITAL SIGNALS," which is 
incorporated by reference herein, to the 8- VSB terrestrial broadcast format of the ATSC 

standard described above. 

[0047] Applying layered modulation to the 8-VSB signal, embodiments of the present 
invention introduce a new signal with a reduced power (a lower layer VSB signal) 
transmitted simultaneously over the legacy 8-VSB signal. The legacy signal can now be 
viewed as an upper layer signal in a layered modulation system. Existing receivers will 
continue to receive the upper layer signal without being "aware" of the presence of the new 
signal; the lower layer signal is ignored as "noise" in legacy receivers. However, new 
layered modulation VSB receivers can receive and process both the new lower layer signal 
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and the legacy signal. The result is an increased throughput for the new layered modulation 
VSB receivers. Although layered modulation increase spectral efficiency over the allocated 
bandwidth, it generally requires a higher carrier to noise ratio (CNR) for the legacy VSB 
signal in order to insert the lower layer VSB signal. 

[0048] As terrestrial broadcasting inherently carries a higher CNR at closer distances to the 
transmitter (following 1/R rule and an antenna receive pattern), in some embodiments, the 
new lower layer VSB signal may be designed with a shorter maximum range than that of the 
legacy VSB signal, if it is desirable to limit the impact of the new VSB service on the 
maximxmi range performance of the legacy VSB service. This can be achieved through 
several techniques. For example, one technique is to reduce the height of the transmit 
antenna, resulting in a shorter horizon range. Another techniques is to point the antenna with 
a larger depression angle, avoiding most energy from the lower layer signal into the legacy 
signal at long ranges. Still another technique is to generally re-shape the antenna elevation 
pattem to provide the desirable power separation between the legacy VSB signal and the new 
lower layer VSB signal and minimize the impact on the legacy signal operation. 

[0049] FIGS. 4A-4D illustrates the relationship between the upper layer legacy 8-VSB 
symbol constellation 400 and the lower-layer 2- VSB signal. FIG. 4A illustrates the upper 
layer 8-VSB symbol constellation 400 alone which is generated as previously discussed. The 
8-VSB nodes are equally spaced on the in-phase (I) axis by themselves. Only the processed 
in-phase component is shown in the diagrams because the quadrature (Q) component does 
not carry information. FIG. 4B illustrates the new lower layer 2-VSB symbol constellation 
superimposed over the upper layer 8-VSB symbol constellation 400. Adding the 2-VSB 
signal perturbs the eight upper layer nodes with uncertainties proportional to the strength of 
the lower layer 2-VSB signal relative to that of the 8-VSB signal. FIG. 4C illustrates the 
influence of the lower layer 2-VSB signal as perceived by a legacy 8-VSB receiver (and in 
the upper layer demodulation of new receiver). The lower layer 2-VSB signal appears as 
additional noise varying the positions of the 8-VSB node positions to appear as ranges 402 to 
the legacy receiver. However, the eight levels of the legacy signal can still be distinguished 
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so the signal is still acceptable. On the other hand, in a new receiver, the ideal 8-VSB signal 
is reconstructed and subtracted from the received signal to expose the 2-VSB signal for 
further lower layer processing. FIG. 4D illustrates the 2-VSB symbol constellation 404 after 
successftil subtraction of the upper layer 8-VSB signal from the composite signal. Note that 
the upper and lower layer signals are designed to occupy a common frequency band. That is, 
they interfere within the common frequency band. However, the nominal frequencies of the 
upper and lower layer signals need not be identical. In fact, the lower layer signal frequency 
can be offset from the upper layer signal frequency to improve performance although the 
signals still interfere with one another. 

[0050] It should also be noted that the lower layer signal may be implemented with other 
modulation formats, e.g. QPSK, so long as the signal remains contained within the designated 
bandwidth. 

[0051] FIG. 5 illustrates the geometry used to calculate anterma gain for exemplary 
performance curves. A 4/3 Earth model is adopted to account for the fact that an 
electromagnetic wave propagates through air layers of different densities and diffracts in a 
downward fashion. Thus, the effective Earth radius, as perceived by the wave, is 4/3 times 
the actual radius of 3960 mi, or 5280 mi. The radio horizon range is approximately 
proportional to the square root of the transmit anteima height. For example, an anterma 1/4 of 
the height would reach 1/2 of the horizon range. 

[0052] In the examples discussed below, the antennas transmitting the respective upper and 
lower layer signals are assumed to point to their respective radio horizons for maximum 
range performance. For simplicity in the analysis, the near-in antenna gain pattern is 
assumed to be quadratic in dB as a fiinction of angle from the anterma boresight. As shown 
in FIG. 5, hu is the height of the upper layer signal antenna (for transmitting the 8-VSB 
legacy signal), hi is the height of the lower layer signal antenna (e.g. for transmitting a 2- 
VSB signal). Ru is the radio horizon range to the upper layer signal antenna and Ri is the 
radio horizon range to the lower layer signal antenna. Re is the effective Earth radius, 5280 
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miles with the 4/3 Earth model, and Oi is the geocentric angle to the ground point, P. The 
following equations relate the geometry of FIG. 5. 

R^^^IRJ^ (1) 

R, « 42RX (2) 



r^=R^sm9, 



(4) 



a: = tan"^ ^ 



(5) 



Thus, the elevation angle to the bore-sight Oi (as measured from the horizontal line through 
the antenna) can be calculated for the ground point P at geocentric angle Of, The antenna 
bore-sight angle Oi can be used to calculate the angle off the bore-sight and therefore the 
antenna gain at Point P. 

[0053] FIGS. 6 A & 6B illustrate examples for CNR with the insertion of a lower layer VSB 
signal over the legacy 8-VSB signal. The examples are not intended to impose specific 
antenna pattern requirements for the invention method. Rather, they serve to demonstrate 
how the invention might work. 

[0054] FIG. 6A plots an example scenario with the CNR values from two antenna heights 
of 2000 and 450 ft with corresponding horizon ranges of 63 and 30 mi. This example has the 
upper layer antenna hit a horizon range with a CNR of 1 8 dB which is the assumed 
requirement for ATSC VSB-8 modulation. Similarly, the lower layer antenna hits a horizon 
range with a CNR of 5dB which is the assumed requirement for a 2-VSB modulation with 
turbo coding and a code rate of 3/4. The CNR curves are principally determined by two 
factors: the 1/R from range effect and the antenna gain off bore-sight. For simphcity, the 

14 

PD-201062 

109.79-US-Ul 



antenna gain is assumed to have such a shape that it combines with the 1/R^ factor to result in 
a constant CNR at near-in range. Since the lower layer carrier power and thermal noise 
combine to form the perceived noise floor for upper layer signal, the CNR for the upper layer 
signal is the difference between the upper signal power and this noise floor, as shown in FIG. 
6 A. The CNR for the upper layer signal achieves the required value of 1 8 dB up to its 
horizon range of 63 mi. Similarly, the CNR for the lower layer signal achieves the required 
value of 5 dB up to its horizon range of 30 mi. The lower layer signal does not reduce the 
maximum range of the upper layer signal. 

[0055] FIG. 6B shows a different example with a required CNR of 1 0 dB for the lower 
layer signal. In this case more throughput from the new service is provided than the previous 
example of FIG. 6A. All other parameters being unchanged from those of FIG. 6 A, the 
maximum range for the lower layer signal is shown to be reduced to approximately 16 mi., 
the horizon range for the designed antenna height of 125 ft. Performance of the legacy 
receiver remains unchanged. 

[0056] FIG. 7 illustrates an exemplary system architecture of the invention. The system 
700 includes one or more transmitters 702 for transmitting both the legacy upper layer 8-VSB 
signal 704 A and the new lower layer signal 704B. The upper layer signal 704 A is 
transmitted from a first antenna 706A and the lower layer signal 704B is transmitted from a 
second antenna 706B. The second antenna 706B for the lower layer signal 704B can be 
disposed a lower height than the first antenna 706 A in order to provide a selective range limit 
716B of the lower layer signal 704B to be less than the range limit 716A of the upper layer 
signal 704A. The lower layer signal range limit 716B is selectively limited so that the lower 
layer signal 704B does not interfere with the upper layer signal 706B in a range 718 where 
the lower layer signal 704B could not be received anyway (no or little physical transmitter 
illumination beyond 716B). Thus, by applying the selective range limit 716B to the lower 
layer signal 704B, the upper layer signal 704A can still be received out to its range limit 
716A. Alternately (or in conjunction with the lower height of the second antenna 706B), the 
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selective range limit 716B can be achieved through the use of a shaped antenna known to 
those skilled in the art. 

[0057] Thus, the layered signal, comprising both the upper layer signal 704 A and the lower 
layer signal 704B, is transmitted only out to the selective range limit 716B (the second 
anteima 706B range of the lower layer signal 704B). Within this range limit 716B, a first 
legacy antenna 708 A and legacy receiver 712A can receive and decode the upper layer legacy 
signal 704A from the layered signal (ignoring the lower layer signal 704B as noise). In 
addition, a layered signal antenna 710 and layered signal receiver 714 can receive and decode 
both the upper layer signal 704A and the lower layer signal 704B within this selective range 
limit 716B. Beyond this selective range limit 716B and out to the upper layer signal rang 
limit 716A, a second legacy antenna 708B and legacy receiver 708B can still receive and 
decode the upper layer signal 704A. 

[0058] Some further design factors should be considered in order to maximize the overall 
range performance of the system 700. In some embodiments of the invention the lower layer 
antenna height can be reduced. This results in a shortened horizon range, which avoids 
potential performance impact on the long range upper layer signal within the horizon of the 
lower layer signal antenna as related to the discussion in the examples of FIGS. 6 A & 6B 
above. In other embodiments, the upper layer signal power can be increased. This leaves 
more "breathing room" for the lower layer signal power to grow, which can in tum support a 
longer horizon (by raising the lower layer antenna height) or service range for the lower layer 
service. 

[0059] FIGS. 8 A & 8B are alternate block diagrams for reception of a layered modulation 
signal by a typical layered receiver 714 of the present invention. FIG. 8 A depicts reception 
where layer subtraction is performed on a signal after the upper carrier has been 
demodulated. The upper layer 8-VSB signal of the received layered signal 816 is processed 
by a demodulator 804 to produce the stable demodulated signal output 820. The 
demodulated signal is fed to a decoder 802 which decodes the upper layer. The upper layer 
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decoder 802 produces the upper layer symbols which are output to an upper layer transport 
and also used to generate an idealized upper layer signal. The upper layer symbols may be 
produced from the decoder 802 for example after Reed-Solomon (RS) decode (BER<10'^ or 
so), in typical decoding operations for the ATSC 8-VSB legacy signal 704A. The upper layer 
symbols from the upper layer decoder 802 are fed to an re-encoder/remodulator 806 which 
effectively produces an idealized upper layer signal for subtraction from the stable 
demodulated signal 820. 

10060] In order for the subtraction to leave a suitable lower layer signal 704B, the upper 
layer legacy signal 704A must be precisely reproduced. The modulated signal may have 
been distorted, for example, by amplifier non-linearity in transmission. These and other 
distortion effects can be estimated from the received signal after the fact or from amplifier 
characteristics which can be downloaded into the receiver in AM - AM and/or AM - PM 
maps 818, which are used to eliminate the distortion. 

[0061] A subtractor 812 then subtracts the idealized upper layer signal from the stable 
demodulated signal 820. This leaves the lower layer signal 704B. The subtractor includes a 
buffer or delay function to retain the stable demodulated signal 820 while the idealized upper 
layer signal 704A is being reconstructed. The lower layer signal 704B is demodulated 810 
and decoded 808 according to its signal format. 

[0062] FIG. 8B depicts reception where alternately layer subtraction is performed on the 
received layered signal. In this case, the demodulator 804 produces the carrier of the upper 
layer legacy 8-VSB signal 704A (as well as the stable demodulated signal output 820). The 
upper layer carrier signal is output to the re-encoder/remodulator 806, which effectively 
produces an idealized upper layer signal which includes the upper layer carrier for subtraction 
from the received combined signal 816. Other equivalent methods of layer subtraction will 
occur to those skilled in the art and the present invention should not be limited to the 
examples provided here. Furthermore, those skilled in the art will understand that the present 
invention is not limited to two layers; additional layers may be included. Idealized upper 
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layers are produced through re-encoding/remodulation from their respective layer symbols 
and subtracted. Subtraction may be performed on either the received combined signal or a 
demodulated signal. Finally, it should be noted that it is not necessary for the lower layer 
signal to be a digital transmissions, particularly 2-VSB); the lower layer signal may be an 
analog transmission or other digital formats as well. 

[0063] FIG. 9A illustrates an exemplary transmission method 900 of the invention. At step 
902, an upper layer signal is transmitted with a first antenna to at least one receiver, the upper 
layer signal comprising an 8-VSB signal. At step 904, a lower layer signal is transmitted 
with a second antenna to the at least one receiver, wherein a layered modulation signal 
comprises the upper layer signal and the lower layer signal both interfering in a common 
frequency band such that the at least one receiver demodulates the upper layer signal directly 
from the layered modulation signal and demodulates the lower layer signal after subtracting 
the upper layer signal from the layered modulation signal. The method 900 can be further 
modified consistent with the transmission system previously described. 

[0064] FIG. 9B illustrates an exemplary receiving method 940 of the invention. At step 
942, an upper layer signal comprising an 8-VSB signal is demodulated directly from a 
layered modulation signal, the layered modulation signal comprising both the upper layer 
signal and a lower layer signal both interfering in a common frequency band. At step 944, 
the lower layer signal is demodulated after subtracting the upper layer signal from the layered 
modulation signal. The method 940 can be fiirther modified consistent with the receiver 
systems previously described. 

[0065] This concludes the description including the preferred embodiments of the present 
invention. The foregoing description of the preferred embodiment of the invention has been 
presented for the purposes of illustration and description. It is not intended to be exhaustive 
or to limit the invention to the precise form disclosed. Many modifications and variations are 
possible in light of the above teaching. 
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[0066] It is intended that the scope of the invention be Hmited not by this detailed 
description, but rather by the claims appended hereto. The above specification, examples and 
data provide a complete description of the manufacture and use of the apparatus and method 
of the invention. Since many embodiments of the invention can be made without departing 
from the scope of the invention, the invention resides in the claims hereinafter appended. 
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